3-dimensional nand flash layer variation aware ssd raid

ABSTRACT

An apparatus is disclosed having a parity buffer having a plurality of parity pages and one or more dies, each die having a plurality of layers in which data may be written. The apparatus also includes a storage controller configured to write a stripe of data across two or more layers of the one or more dies, the stripe having one or more data values and a parity value. When a first data value of the stripe is written, it is stored a a currant value in a parity page of the parity buffer, the pants page corresponding to the stripe. For each subsequent data value that is written, an XOR operation is performed with the subsequent data value and the current value of the corresponding parity page and the result of the XOR operation is stored as the current value of the corresponding parity page.

BACKGROUND Field of the Disclosure

This disclosure relates to data storage devices. More particularly, thedisclosure relates to writing data to 3-dimensional NAND (3D NAND) flashstorage devices.

Description of the Related Art

Data storage devices may be used to store data used by computingdevices. The data that is stored on the data storage devices may becomeinaccessible, corrupted, damaged, or may have errors. Various errorcorrection and/or detection schemes, codes, algorithms, functions,operations, etc., maybe used to protect the data that is stored on thedata storage devices, from loss. For example, when writing to a datastorage device, a storage controller may utilize error correction codes(ECC) to detect and correct errors in the writing process.

SUMMARY

In some implementations, the present disclosure relates to a method. Themethod includes writing a stripe of data across two or more layers ofone or more dies, the stripe having one or more data values and a parityvalue. Responsive to writing a first data value of the stripe, the firstdata value is stored as a current value in a parity page of a paritybuffer. The parity page corresponds to the stripe. Responsive to writingeach subsequent data value of the stripe, an XOR operation is performedwith the subsequent data value and the current value of thecorresponding parity page and the result is stored as the current valueof the corresponding parity page.

In some implementations, the present disclosure relates to an apparatus.The apparatus includes a parity buffer having a plurality of paritypages, one or more dies, each die having a plurality of layers in whichdata may be written, and a storage controller configured to write astripe of data across two or more layers of each of the one or moredies, the stripe having one or more data values and a parity value.Responsive to writing a first data value of the stripe, the controllermay store the first data value as a current value in a parity page ofthe parity buffer, the parity page corresponding to the stripe.Responsive to writing each subsequent data value of the stripe, thecontroller may perform an XOR operation with the subsequent data valueand the current value of the corresponding parity page find store aresult as the current value of the corresponding parity page.

In some implementations, the present disclosure relates to anon-transitory machine-readable medium having executable instructions tocause one or more processing devices to perform a method. The methodcomprising writing a stripe of data across two or more layers of one ormore dies, the stripe having one or more data values and a parity value.Responsive to writing a first data value of the stripe, the first datavalue is stored as a current value in a parity page of a parity buffer.The parity page corresponds to the stripe. Responsive to writing eachsubsequent data value of the stripe, an XOR operation is performed withthe subsequent data value and the current value of the correspondingparity page and the result is stored as the current value of thecorresponding parity page.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example data storage system, inaccordance with one or more embodiments of the present disclosure.

FIG. 2A is a diagram illustrating an example 3D NAND architecture, inaccordance with one or more embodiments of the present disclosure.

FIG. 2B is a diagram illustrating a circuit representation of theexample 3D NAND architecture of FIG. 2A, in accordance with one or moreembodiments of the present disclosure.

FIG. 3 is a diagram illustrating an example charge trap memory cell, inaccordance with some embodiments of the present disclosure.

FIG. 4 is a graph illustrating example RBER rates across layers for 2separate memory dies, in accordance with one or more embodiments of thepresent disclosure.

FIGS. 5A-5D are block diagrams illustrating a logical layout of anexample data storage device in accordance with one or more embodimentsof the present disclosure.

FIG. 6 is a flow diagram illustrating a method of writing a data stripe,in accordance with one or more embodiments of the present disclosure.

FIG. 7 is a block diagram of a non-volatile data storage system, inaccordance with one or more embodiments of the present disclosure.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures. It is contemplated that elements disclosed in oneembodiment may be beneficially utilized on other embodiments withoutspecific recitation.

DETAILED DESCRIPTION

In the following disclosure, reference is made to examples,implementations, and or embodiments of the disclosure. However, itshould be understood that the disclosure is not limited to specificdescribed examples, implementations, and/or embodiments. Any combinationof the features, functions, operations, components, modules, etc.,disclosed herein, whether related to different embodiments or not, maybe user to implement and practice the disclosure. Furthermore, althoughembodiments of the disclosure may provide advantages and or benefitsover other possible solutions, whether or not a particular advantage andor benefit is achieved by a given embodiment is not limiting of thedisclosure. Thus, the following aspects, features, embodiments andadvantages are merely illustrative and are not considered elements orlimitations of the appended claims except where explicitly recited in aclaim(s). Likewise, reference to “the disclosure” shall not be construedas a generalization of any inventive subject matter disclosed herein andshall not be considered to be an element or limitation of the appendedclaims except where explicitly recited in the claim(s).

The headings provided herein are for convenience only and do notnecessarily affect the scope or meaning of the claimed invention.Disclosed herein are example implementations, configurations, and orembodiments relating to storing neural networks and weights for neuralnetworks.

Data storage devices, such as solid state drives (SSDs), hard diskdrives (HDDs), hybrid drives (e.g., storage drives/devices that includeboth magnetic media medium and flash memory), etc., typically includeone or more controllers coupled with one or more non-volatile memory(NVM) arrays. Stored data may be subject to loss and or corruption. Forexample, data may be lost, damaged, corrupted, etc., due to failure ofmemory cells, damage (e.g., physical damage), degradation, read writedisturbs, loss of data retention, loss of endurance, etc. Data storagedevices may generally utilize one or more error correction codes (ECCs)and or error coding mechanisms to detect and or correct errors in thedata that is stored within the data storage devices (e.g., stored withinthe NVM arrays). Some ECC are tunable to adapt to media errorcharacteristics. The required strength of the ECC depends on, amongother factors, the residual bit error rate (RBER) and the endurance ofthe SSD, also known as the end of life (EoL). More specifically, as anSSD reaches its EoL (and thus, the media characteristics wear down), theRBER of the media will increase thus requiring a stronger ECC. As thestrength of the required ECC increases, the computational resources andor storage space used by the data storage device also increase. Forexample, a stronger ECC may use more computational resources (e.g., morecomputing power) to decode codewords. In another example, a strong ECCmay use more storage space (e.g., more bits, bytes, etc., of data) tostore codewords generated using the stronger ECC. Redundant Array ofIndependent Disks (RAID) involves the use of multiple disks (or, dies)and may be used when the RBER has increased beyond an ECC's errorcorrection capability.

In 3D NAND memory, the RBER can differ from die to die, layer to layerand even from the most significant hit (MSB) to the least significantbit (LSB) of a word line of a layer. However, default ECC coding schemesselect the strength of the ECC based on RBER of the layer with the worstRBER. Although a data stripe may be written to a 3D NAND memory in sucha way that the layer to layer RBER variation is averaged acrossdifferent RAID groups, in order to avoid disrupting the programmingsequence, a word line in each chip must be left blank. This results in acomputational overhead and or storage space overhead which isundesirable.

FIG. 1 is a diagram illustrating an example data storage system 100, inaccordance with some embodiments of the present disclosure. The datastorage system 100 includes a computing device 110 and a data storagedevice 120. The computing device 110 may also be referred to as a hostsystem. In one embodiment, the data storage device 120 may be part ofthe computing device 110 (e.g., may be located inside of a housing,chassis, case, etc., of the computing device 110). In another example,the data storage device 120 may be separate from the computing device110 (e.g., may be an external device that is coupled to the computingdevice 110 via a cable, such as a universal serial bus (USB) cable).

The data storage device 120 may incorporate access command schedulingand or execution in accordance with embodiments, examples, and orimplementations disclosed herein. The data storage device 120 may be anytype of data storage device, drive, module, component, system, or thelike. Furthermore, the terms “drive” and “data storage drive” may beused herein in certain contexts to refer to any type of data storagedevice, and may be used substantially interchangeably with the term“data storage device” herein in connection with various embodiments andor in various contexts. As shown, the data storage device 120 (e.g.,hybrid hard drive, solid-state drive, any storage device utilizingsolid-state memory, etc.) includes a controller 130 (e.g., controlcircuitry, software, firmware, or a combination thereof) andnon-volatile memory 140.

The non-volatile memory (NVM) 140 may be configured for long-termstorage of data and may retain data between power on/off cycles of thedata storage device 120. The non-volatile memory 140 and or portions ofthe non-volatile memory 140 may also be referred to as a storage medium.In some embodiments, the non-volatile memory 140 may include solid-statememory. Solid-state memory may comprise a wide variety of technologies,such as 3D NAND memory or 2D NAND memory (e.g., single-level cell (SLC)memory, multi-level cell (MLC) memory, triple level cell (TLC) memory,X4 or quad-level cell (QLC) memory, etc.). The non-volatile memory 140may include any combination of the one or more types of memoriesdescribed here. The non-volatile memory 140 may be divided logicallyand/or physically into dies (or, chips), layers, word lines, army's,planes, blocks, pages, tracks, and sectors.

The controller 130 may include one or more processors, memory devices,data and or power transmission channels/paths, boards, or the like. Insome embodiments, the controller 130 may be implemented as one or moresystem-on-a-chip (SoC) modules, field-programmable gate array (FPGA)modules, application-specific integrated circuit (ASIC) modules,processing devices (e.g., processors), chips, or the like. In otherembodiments, one or more components of the controller 130 may be mountedon a printed circuit board (PCB). The controller 130 may be configuredto receive data commands from a storage interface (e.g., a devicedriver) residing on the computing device 110. FIG. 1 illustratescontroller 130 having a local memory 131, and module 131A storedtherein. Controller 130 may execute module 131A to perform theoperations described herein with respect to FIGS. 5A-5D and 6.

The controller 130 may also include a parity buffer 170. Parity buffer170 may comprise a storage class memory (also known as persistentstorage memory). Because it is persistent, parity buffer 170 may retainits data even during a power cycle. As described in further detailherein, parity buffer 170 may be organized into a number of pages foruse in writing stripes of data.

The controller 130 may communicate with the computing device 110 over ahost interface 160, and may receive commands via the host interface 160.These commands may be referred to as data commands, data accesscommands, data storage access commands, etc. Data commands may specify ablock address in the data storage device 120. Data may be accessedtransferred based on such data commands. For example, the controller 130may receive data commands (from the computing device 110) and mayexecute such commands on in the non-volatile memory 140 (e.g., in one ormore arrays, pages, blocks, sectors, etc,). The data commands receivedfrom computing device 110 may include read data commands, write datacommands, and erase data commands. The controller 130 may be coupled tothe non-volatile memory (NVM) 140 via a NVM interface 150. In oneembodiment, the NVM interface 150 may include a plurality of channels(e.g., one or more lines, pines, wires, traces, etc.) and each channelmay be coupled to different portions of the non-volatile memory 140(e.g., different NVM arrays, different flash arrays, etc.).

The controller 130 may execute the received data commands to read,write, and erase data from non-volatile memory 140, via the NVMinterface 150. For example, the commands may include a read command(e.g. a data read command) to read a block of data from the non-volatilememory 140. The controller 130 may read the data from the page and maytransmit the data to the computing device 110 via the host interface160. In another example, the commands may include a write command (e.g.,a data write command) to write data to a page in a non-volatile memory140. In one embodiment, write commands may include program commands(e.g., a command to write the value “1” to a location the non-volatilememory 140) and erase commands (e.g., a command to write the value “0”to a location, a page, a block, etc., in the non-volatile memory array).The controller 130 may receive the data from the computing device 110via the host interface 160 and may write the data to the page. The hostinterface 160 may include hardware (e.g., wires, pins, traces,connectors, etc.), software (e.g., drivers), firmware, or a combinationthereof, that allows the processing device 111 and or the computingdevice 110 to communicate data with the data storage device 120.Examples of a host interface may include a peripheral componentinterconnect express (PCIe) bus, a serial AT attachment (SATA) bus, anon-volatile memory express (NVMe) bus. etc,

The data storage device 120 may store data received from the computingdevice 110 such that the data storage device 120 acts as data storagefor the computing device 110. To facilitate this function, thecontroller 130 may implement a logical interface. The logical interfacemay present to the computing device memory a set of logical addresses(e.g., sequential/contiguous addresses) where data may be stored.Internally, the controller 130 may map logical addresses to variousphysical memory addresses in the non volatile memory arrays and or othermemory module(s). Mapping data indicating the mapping of logicaladdresses to physical memory addresses may be maintained in the datastorage device. For example, mapping table data may be stored innon-volatile memory 140 in order to allow for recreation of mappingtables following a power cycle.

The controller 130 may encode data when storing the data on thenon-volatile memory 140. The controller 130 may encode the data toprotect the data from errors, loss, corruption, etc. The controller 130may protect the data from errors, loss, corruption, etc., using variousmethods, techniques, functions, operations, actions, etc. In oneembodiment, the controller 130 may protect the data by generating paritydata (e.g., parity bits). The parity data may allow the controller 130to determine whether there am errors in the data (e.g., errors due tocorruption, damaged cells, damaged blocks, error while leading the data,etc.). The parity data (e.g., one or more parity bits) may be generatedusing various algorithms, techniques, functions, operations, etc. Inanother embodiment, the controller 130 may use an ECC to generatecodewords. The codewords may also allow the controller 130 (e.g., thedecoder 132) to correct or recover from errors in the codewords.

The controller 130 may also decode data that is stored on thenon-volatile memory 140. In one embodiment, controller 130 may decodecodewords which encode the data that is stored on the non-volatilememory 140. In another embodiment, the controller 130 may perform errordetection to determine the integrity of data retrieved from non-volatilememory 140 (e.g., to determine whether the data has errors). Forexample, the controller 130 may use parity data to check the data todetermine whether there is an error in the data (e.g., whether one ormore bits in the data are incorrect due to corruption, damage cells,damaged blocks, etc.). In 3D NAND memory, the RBER can differ from dieto die, layer to layer and even from the most significant bit (MSB) tothe least significant bit (LSB) of a word line of a layer. Becausedefault ECC coding schemes select the strength of the ECC based on RBERof the layer with the worst RBER, a stronger ECC is generally required.As the strength of the required ECC increases, the computationalresources and/or storage space used by the data storage device alsoincreases. Embodiments of the present disclosure include distributing adata stripe across a word line of each layer of each die in non-volatilememory 140. In this way, the RBER variation seen between the MSB and theLSB of word lines, between layers, and between dies may be averagedacross all the values of a stripe thereby allowing controller 130 todetermine an ECC for the stripe based on the average RBER, instead ofthe worst case RBER. This results in a weaker ECC and saving ofcomputational resources and/or storage space.

FIG. 2A shows a memory structure of a block 200 of 3D NAND memory, whichmay correspond to memory 140 (illustrated in FIG. 1). More specifically,FIG. 2A illustrates a cross section along the bit line direction (alongy-direction) in which straight vertical NAND strings (depicted as siring0 and string 1) extend from common source connections in or near asubstrate to global bit lines (GBLO-GBL3) that extend over the physicallevels of memory cells. FIG. 2A also illustrates layers L0-LN and thecorresponding word lines WL0-WLN in a given physical level in a block.The word lines WL0-WLN are formed from a sheet of conductive material.FIG. 2A also illustrates bit lines BLO-BL3, each of which comprise amemory hole structure having a substrate and a charge trap (shown inFIG. 3) that extend down through these sheets of conductive materialthereby forming memory cells 205 where a bit line and word lineintersect (as further illustrated in FIG. 3). The memory cells arcconnected in series vertically (along the z-direction) by the verticalbit lines BLO-BL3. The sheet of conductive material may comprise acontrol gate of each memory cell formed in this way and thus the controlgates of cells in the same layer (along the Y axis) may be linked toform a word line. FIG. 3 illustrates a cross-section of a memory cell210 in accordance with some embodiments. As discussed above, the memorycell 210 may comprise a control gate 215 (e.g., the conductive materialfrom a word line illustrated in FIG. 2A) and a memory hole structure 220having a substrate 220A and a charge trap 220B (e.g. a bit line of FIG.2A). Referring back to FIG. 2A, each bit line BL0-BL3 now connectstogether 1 charge trap cell from each layer of the chip. For example,within a given block there are multiple NAND strings connected to agiven global bit line (e.g. GBLO connects with multiple BLOs). NANDstrings are grouped into sets of strings that share common select lines.Thus, for example, NAND strings that are selected by source select lineSGSO and drain select line SGDO may be considered as a set of NANDstrings and may be designated as String 0, while NAND strings that areselected by source select line SGS1 and drain select line SGD1 may beconsidered as another set of NAND strings and may be designated asString 1 as shown. A block may consist of any suitable number of suchseparately-selectable sets of strings. It will be understood that FIG.2A shows only portions of GBLO-GBL3, and that these bit lines extendfurther in the y-direction and may connect with additional NAND stringsin the block and in other blocks. Furthermore, additional bit linesextend parallel to GBLO GBL3 (e.g. at different locations along x-axis,in front of, or behind the location of the cross-section of FIG. 2A).

FIG. 2B illustrates separately-selectable sets of NAND strings of FIG.2A schematically. It can be seen that each of the global bit lines(GBLO-GBL3) is connected to multiple separately selectable sets of NANDstrings (e.g. GBLO connects to vertical bit line BLO of String 0 andalso connects to vertical bit line BLO of String 1) in the portion ofthe block shown. In some eases, word lines of all strings of a block areelectrically connected, e.g. WLO in string 0 may be connected to WLO ofString 1, String 2, etc. Such word lines may be formed as a continuoussheet of conductive material that extends through all sets of strings ofthe block. Source lines may also be common for all strings of a block.For example, a portion of a substrate may be doped to form a continuousconductive region underlying a block that is isolated from similarconductive regions underlying other blocks thus allowing separatebiasing to erase a block as a unit Source and drain select lines are notshared by different sets of strings so that, for example. SGDO and SOSOcan be biased to select String 0 without similarly biasing SGD1 andSGS1. Thus, String 0 may be individually selected (connected to globalbit lines and a common source) while String 1 (and other sets ofstrings) remain isolated from global bit lines and the common source.Accessing memory cells in a block during programming and readingoperations generally includes applying select voltages to a pair ofselect lines (c.g. SGSO and SGDO) while supplying unselect voltages toall other select lines of the block (c.g. SCSI and SGDI). Then,appropriate voltages are applied to word lines of the block so that aparticular word line in die selected set of strings may be accessed(c.g. a read voltage is applied to the particular word line, whileread-pass voltages are applied to other word lines). Erasing operationsmay be applied on an entire block (all sets of strings in a block)rather than on a particular set of strings in a block.

FIG. 4 illustrates a graph 400 of example residual bit error rate (RBER)across the layers of 2 separate dies. The graph 400 shows the RBER overthe least significant bit (LSB) and most significant bit (MSB) of 2separate memory dies. Graph 400 shows plots for the RBER variation overlayers for the most significant bit and least significant bit of a firstdie (MSB die 1 and LSB die 1) and the most significant bit and leastsignificant bit of a second die (MSB die 2 and LSB die 2). As can beseen, the RBER varies greatly from layer to layer and from MSB to LSB inthe word line for each layer. Because of this variation, traditionallythe required strength of the ECC is based on the worst RBER (depicted asMAX MSB in FIG. 3). However, as discussed further herein, this layer tolayer and MSB to LSB RBER variation may be averaged across the datavalues in each stripe to be written, such that each stripe has an RBERthat is an average of the RBER across the layers. In this way, thestrength of the ECC required to w rite a stripe of data is reduced, asthe ECC is now selected based on the average RBER, instead of the worstRBER.

FIG. 5A illustrates a logical layout 500 of a data storage device, suchas data storage device 120 of FIG. 1. The logical layout 500 may also bereferred to as a logical arrangement. In the example shown in FIG. 5A,non-volatile memory 140 comprises 8 dies D0-D7, and 12 pages per die,however non-volatile memory 140 may comprise any appropriate number ofdies and pages per die. Each die may have layers L0-L5, and each layermay include a word line (not shown) including a most significant bit anda least significant bit. FIG. 5A also illustrates parity buffer 170,having a plurality of parity pages. The number of parity pages in paritybuffer 170 may correspond to the smaller value between the number ofpages per die or 2N-1, where N is the stripe size (i.e. number of valuesin a stripe including data and parity). In response to receiving arequest to write data stripes 1-8 (e.g., from computing device 110),controller 130 may write data stripes 1-8 to the non-volatile memory 140as discussed herein. As shown in FIGS. 5A-5D, the data values of eachstripe may be represented by the number of that stripe, while the parityvalue for each stripe may be represented by “P.” For example, each datavalue in stripe 1 may be denoted by “1” and the parity value 505 ofstripe 1 may be denoted by “P1.” FIG. 5A illustrates the location of theparity values for each stripe 1-8. As can be seen, controller 130 maywrite the first value of each stripe across the 8 dies. Upon writing thefirst data value of stripe 1 (“1” and denoted as 501), controller 130may store this value as the current value of a parity page 506 of theSCM parity buffer 170 corresponding to stripe 1. Similarly, upon writingthe first value of stripe 2 (“2”), controller 130 may store this valueas the current value of a parity page 507 corresponding to stripe 2.

FIG. 5B illustrates the logical layout 500 after additional values fromeach stripe have been written. As illustrated in FIG. 5B, the datavalues of each stripe arc written across each layer L0-L5 of each dieD0-D7, and across a word line of each layer. To write data values of astripe across a word line, the controller 130 may write a first datavalue of a stripe (e.g., 501) into the most significant bit page of aword lino corresponding to L0 (MSB0) on D1 and write a second data valueof the stripe (e.g., 502) into the least significant bit page of theword line corresponding to L0 (LSB0) on D1. Controller 130 may write athird data value of the stripe into the MSB1 of the word linecorresponding to L1 on D2, and write a fourth data value of the stripeinto the LSB1 of the word line corresponding to L1 on D2. For each datavalue of stripe 1 that is written, controller 130 may perform an XORoperation with that data value and the current value of the parity page506 and store the result as the new current value of the parity page506. For example, in response to writing the second data value (e.g.,502) of Stripe 1, controller 130 may perform an XOR operation with thesecond data value 502 and the current value of the parity page 506 (inthis case, first data value 501) and store the result as the currentvalue of the parity page 506. Upon writing the third data value 503 ofStripe 1, controller 130 may perform an XOR operation with the thirddata value 503 and the current value of the parity page 506 and storethe result (shown in FIG. 5B as “X”) as the current value of the paritypage 506. Controller 130 may continue in this fashion until each datavalue of stripe 1 is written and may write each stripe 1-8 in a similarfashion as described above.

FIG. 5C illustrates the logical layout 500 after all of the data valuesof Stripe 1 have been written. Upon writing the final data value 504 ofstripe 1, controller 130 may perform an XOR operation with the finaldata value 504 and the current value of the parity page 506 (the paritypage corresponding to Stripe 1) and store the result (shown in FIG. 5Cas “P”) as the current value of the parity page 506 corresponding toStripe 1. Controller 130 may then write the current value of the paritypage 506 to the location of the parity value 505 (denoted in FIG. 5C as“P1”) for stripe 1 (i.e. the parity value 505 of Stripe 1 may beassigned the current value of parity page 506). Similarly, upon writingthe final data value 508 of stripe 2, controller 130 may perform an XORoperation with the final data value 508 and the current value of theparity page 507 and store the result (shown in FIG. 5C as “P”) as thecurrent value of the parity page 507 corresponding to Stripe 2.Controller 130 may then write the current value of the parity page 507to the location of the parity value 509 (denoted in FIG. 5C as “P2”) forstripe 2 (i.e. the parity value 509 of Stripe 2 may be assigned thecurrent value of parity page 507).

FIG. 5D illustrates the logical layout 500 after all stripes 1-8 havebeen written. As can be seen, every stripe 1-8 is distributed across aword line of each layer L0-L5 of each die D0-D7. In this way, the RBERvariation seen between the MSB and the LSB of word lines, between layersL0-L5, and between dies DQ-D7 may be distributed across all the valuesof a stripe. As a result, the RBER variation seen between the MSB andthe LSB of word lines, between layers L0-L5, and between dies D0-D7 isaveraged over the values of a stripe. Controller 130 may thus determinean ECC for a stripe based on the average RBER, instead of the worst caseRBER, thereby allowing the controller 130 to use a weaker ECC and savingcomputational resources and-or storage space. For example, a weaker ECCmay use less computational resources (e.g., less computing power) todecode codewords. In another example, a weaker ECC may use less storagespace to store codewords generated using the weaker ECC.

FIG. 6 illustrates a (low diagram of a method 600. The method 600 may beperformed by a processing device (e.g., a processor, a centralprocessing unit (CPU), a controller, an application-specific integratedcircuit (ASIC), a field programmable gate array (FPGA), etc.), acomputation engine, etc. For example, the method 600 may be performed bya controller and or a computation engine of a data storage device, suchas controller 130 illustrated in FIG. 1. The controller, processingdevice, and or computation engine may be processing logic that includeshardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (e.g., instructions run on a processor toperform hardware simulation), firmware, or a combination thereof.

At block 605, the controller may write a stripe of data across one ormore layers of one or more dies of NVM 140. The stripe may have one ormore data values and a parity value. More specifically, the controllermay write N pages in parallel to NVM 140, where N is given as:

N−(number of dies)×(number of planes in a die)×(number of pages in awordline)

The controller need not wait for N pages worth of data before writing tothe NVM 140, however. As long as the controller has received enough datafor at least one wordline in a 3D NAND flash memory, it may write to NVM140 directly so long as the controller knows where it will write to next(because 3D NAND supports one shot programming, whereas for 2D NANDflash, the controller writes data in logical page unit). At block 610,responsive to writing a first data value of the stripe, the controllermay store the first data value as a current value in a parity page of aparity buffer. The parity page may correspond to the stripe beingwritten. At block 615, the controller may write a subsequent data value.At block 620, the controller may perform an XOR operation with thesubsequent data value and the current value of the corresponding paritypage. At block 625, the controller may store the result of the XORoperation as the new current value of the corresponding parity page. Atblock 630, the controller may determine whether all the data values arewritten. If there are still data values to be written, the controllerreturns to block 615. If all data values are written, then at block 635the controller writes the current value of the corresponding parity pageto the parity value of the stripe.

FIG. 7 is a block diagram of an exemplary non-volatile data storagesystem 700, which may be similar to data storage system 100 illustratedin FIG. 1. The non-volatile data storage system 700 may include one ormore memory die 702. The memory die 702 includes a memory structure 704of memory cells, such as an array of memory cells herein referred to asa memory array, address controller 706. and read write circuits 730. Thememory structure 704 is addressable by word lines via a row decoder 728and by bit lines via a column decoder 732. The read/write circuits 730include multiple sense blocks SB1, SB2, . . . , SBp (sensing circuitry)and allow a page of memory cells to be read or programmed in parallel.Typically the memory system-based control logic 404 is included in thesame non-volatile memory system 700 (e.g., a removable storage card) asthe one or more memory die 702. Control commands and data (e.g., a rawvideo stream) are transferred between the host system 104 and memorysystem-based control logic 404 via a data bus 718, and between thecontroller and the one or more memory die 702 via lines 716. The databus 718 may for example be a PCIe serial memory bus.

The memory structure 704 can be 2D (laid out in a single fabricationplane) or 3D (laid out in multiple fabrication planes). The memorystructure 704 may comprise one or more array of memory cells including a3D array. In one embodiment, the memory structure 704 may comprise amonolithic three-dimensional memory structure (3D array) in whichmultiple memory levels are formed above (and not in) a single substrate,such as a wafer, with no intervening substrates. The memory structure704 may comprise any type of non-volatile memory that is monolithicallyformed in one or more physical levels of arrays of memory cells havingan active area disposed above a silicon substrate. The memory structure704 may be in a non-volatile memory device (e.g. Non-volatile memorysystem 700) having circuitry associated with the operation of the memorycells, whether the associated circuitry is above or within thesubstrate.

The address controller 706 cooperates with the read write circuits 730to perform memory operations on memory cells of the memory structure704, and includes a state machine 708, an address decoder 710, and apower control 714. The state machine 708 provides control of memoryoperations. A store region selector 712 may be provided, e.g., forprogramming parameters as described further below.

The address decoder 710 provides an address interface between that usedby the host or a memory system-based control logic 404 to the hardwareaddress used by the row decoder 728 and column decoder 732. The powercontrol 714 controls the power and voltages supplied to the variouscontrol lines during memory operations. The power control 714 and orread/write circuits 730 can include drivers for word lines, source gateselect (SGS) transistors, drain gate select (DOS) transistors, bitlines, substrates (in 20 memory structures), charge pumps, and sourcelines. The sense blocks can include bit line drivers and senseamplifiers in one approach.

An SGS transistor is a select gate transistor at a source end of amemory string, and a DOS transistor is a select gate transistor at adrain end of a memory string.

In some implementations, some of the components can be combined. Invarious designs, one or more of the components (alone or incombination), other than memory structure 704, can be thought of as atleast one control circuit or controller which is configured to performthe techniques described herein. For example, a control circuit mayinclude any one of, or a combination of, address controller 706, statemachine 708, address decoder 710, column decoder 732, power control 714,control processor 726, sense blocks SB1, SB2, . . . , SBp, read/writecircuits 730, memory system-based control logic 404, and so forth.

The memory system-based control logic 404 may comprise a controlprocessor 726 and memory devices such as controller read-only memory 720and controller volatile memory 722.

The memory devices of the memory system-based control logic 404 maycomprise code such as a set of instructions, that configure the controlprocessor 726 to execute the set of instructions to provide aspects ofthe functionality described herein. Alternatively or additionally, thecontrol processor 726 can access code from the memory structure 704,such as a reserved area of memory cells in one or more word lines.

For example, code can be used by the memory system-based control logic404 to access the memory structure 704, controller read-only memory 720,or controller volatile memory 722 for partial decoding and/or eventdetection logic. In some embodiments the control logic may utilize thehost system volatile memory 724 for caching some or all of the raw videostream for partial decoding and event detection, using for exampledirect memory access technology (DMA) over the data bits 718. Thecontrol logic may further include drivers to perform basic tasks such ascontrolling and allocating memory, prioritizing the processing ofinstructions, and controlling input and output ports.

Generally, the control logic can include software and other logic (e.g.,circuits, firmware) to perform the functions described herein. Thecontrol logic may comprise a sequencer to control the liming (start andstop times, durations, spacing etc.) of the various signals describedherein. The state machine 708 may also be utilized to implement aspectsof the control logic.

In one embodiment, the host system 104 is a computing device (e.g.,laptop, desktop, smartphone, tablet, digital camera) that includes oneor more processors, one or more processor readable storage devices (hostsystem volatile memory 724. ROM, flash memory, hard disk drive, solidstate memory) that store processor readable code (e.g., software) forprogramming the one or more processors to perform aspects of thetechniques described herein. The host system 104 may also includeadditional system memory, one or more input output interfaces and/or oneor more input output devices in communication with the one or moreprocessors, as well as other components well known in the art.

The store region selector 712 may be a non-volatile memory such as NANDflash memory, or another type. The store region selector 712 identitiesblocks of the memory structure 704 for reading and writing, among otherthings, as known in the an.

One of skill in the art will recognize that the non-volatile memorysystem 700 is an example and that various implementations will includeadditional components, or exclude or combine some of the illustratedcomponents, in manners known in the art.

General Comments

Those skilled in the art will appreciate that in some embodiments, othertypes of distributed data storage systems may be implemented whileremaining within the scope of the present disclosure. In addition, theactual steps taken in the processes discussed herein may differ fromthose described or shown in the figures. Depending on the embodiment,certain of the steps described above may be removed, others may beadded.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of protection. Indeed, the novel methods and systems describedheroin may be embodied in a variety of other forms. Furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made. The accompanying claims and theirequivalents are intended to cover such forms or modifications as wouldfall within the scope and spirit of the protection. For example, thevarious components illustrated in the figures may be implemented assoftware and or firmware on a processor, ASIC/FPGA, or dedicatedhardware. Also, the features and attributes of the specific embodimentsdisclosed above may be combined in different ways to form additionalembodiments, all of which fall within the scope of the presentdisclosure. Although the present disclosure provides certain preferredembodiments and applications, other embodiments that are apparent tothose of ordinary skill in the art, including embodiments which do notprovide all of the features and advantages set forth herein, are alsowithin the scope of this disclosure. Accordingly, the scope of thepresent disclosure is intended to be defined only by reference to theappended claims.

The words “example” or “exemplary” are used herein to mean serving as anexample, instance, or illustration. Any aspect or design describedherein as “example” or “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects or designs. Rather, use ofthe words “example” or “exemplary” is intended to present concepts in aconcrete fashion. As used in this disclosure, the term “or” is intendedto mean an inclusive “or” rather than an exclusive “or” That is, unlessspecified otherwise, or clear from context, “X includes A or B” isintended to mean any of the natural inclusive permutations. That is, ifX includes A: X includes B; or X includes both A and H, then “X includesA or B” is satisfied under any of the foregoing instances. In addition,the articles “a” and “an” as used in this disclosure and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from context to be directed to a singularform. Moreover, use of the term “an embodiment” or “one embodiment” or“an implementation” or “one implementation” throughout is not intendedto mean the same embodiment or implementation unless described as such.Furthermore, the terms “first,” “second,” “third,” “fourth,” etc. asused herein are meant as labels to distinguish among different elementsand may not necessarily have an ordinal meaning according to theirnumerical designation.

All of the processes described above may be embodied in, and fullyautomated via, software code modules executed by one or more generalpurpose or special purpose computers or processors. The code modules maybe stored on any type of computer-readable medium or other computerstorage device or collection of storage devices. Some or all of themethods may alternatively be embodied in specialized computer hardware.

What is claimed is:
 1. An apparatus comprising: a parity bufferconfigured to store a plurality of parity pages: one or more dies, eachdie having a plurality of layers in which data may be written: and astorage controller configured to: write a stripe of data across two ormore layers of the one or more dies, te stripe having one or more datavalues and a parity value: responsive to writing a first data value ofthe stripe, store the first data value as a current value in a paritypage of the parity buffer, the parity page corresponding to the stripe:and responsive to writing each subsequent data value of the stripe,perform an XOR operation with the subsequent data value and the currentvalue of the corresponding parity page and store a result as the currentvalue of the corresponding parity page.
 2. The apparatus of claim 1,wherein to write the stripe, the storage controller is furtherconfigured to: write the stripe across a word line in each of the two ormore layers.
 3. The apparatus of claim 2, wherein to write the stripeacross a word line, the storage controller is further configured to:write a data value of the stripe into a most significant bit page of theword line and write a subsequent data value of the stripe into a leastsignificant bit page of the word line.
 4. The apparatus of claim 1,wherein the storage controller is further configured to: responsive towriting a final data value of the stripe, perform an XOR operation withthe final data value and the current value of the corresponding paritypage and store a result as the current value in the corresponding paritypage: and write the current value in the corresponding parity page tothe parity value of the stripe.
 5. The apparatus of claim 1, wherein thestorage controller is further configured to: write one or moresubsequent stripes of data across the two or more layers of each of theone or more dies, each of the one or more subsequent stripes having oneor more data values and a parity value.
 6. The apparatus of claim 1,wherein a residual bit error rate (RBER) of each of the two or morelayers is averaged across the one or more data values in the stripe, andwherein the average RBER is lower than a worst case RBER.
 7. Theapparatus of claim 6, wherein the storage controller is furtherconfigured to determine an error correction code (ECC) based on theaverage RBER.
 8. The apparatus of claim 1, wherein the parity buffercomprises a persistent storage medium.
 9. A method comprising: writing astripe of data across two or more layers of one or more dies, the stripehaving one or more data values and a parity value: responsive to writinga first data value of the stripe, storing the first data value as acurrent value in a parity page of a parity buffer, the parity pagecorresponding to the stripe; and responsive to writing each subsequentdata value of the stripe, performing an XOR operation with thesubsequent data value and the current value of the corresponding paritypage and storing a result as the current value of the correspondingparity page.
 10. The method of claim 9, wherein writing the stripecomprises writing the stripe across a word line in each of the two ormore layers.
 11. The method of claim 10, wherein writing the stripeacross a word line comprises: writing a data value of the stripe into amost significant bit page of the word line and writing a subsequent datavalue of the stripe into a least significant bit page of the word line.12. The method of claim 9, further comprising: responsive to writing afinal data value of the stripe, performing an XOR operation with thefinal data value and the current value of the corresponding parity pageand storing a result as the current value in the corresponding paritypage: and writing the current value in the corresponding parity page tothe parity value of the stripe.
 13. The method of claim 9, furthercomprising: writing one or more subsequent stripes of data across thetwo or more layers of each of the one or more dies, each of the one ormore subsequent stripes having one or more data values and a parityvalue.
 14. The method of claim 9, wherein a residual bit error rate(RBER) of each of the two or more layers is averaged across the one ormore data values in the stripe, and wherein the average RBER is lowerthan a worst case RBER.
 15. A non-transitory machine-readable mediumhaving executable instructions to cause one or more processing devicesto perform a method, the method comprising: writing a stripe of dataacross two or more layers of one or more dies, the stripe having one ormore data values and a parity value; responsive to writing a first datavalue of the stripe, storing the first data value as a current value ina parity page of a parity buffer, the parity page corresponding to thestripe: and responsive to writing each subsequent data value of thestripe, performing a parity calculation operation with the subsequentdata value and the current value of the corresponding parity page andstoring a result as the current value of the corresponding parity page.16. The non-transitory machine-readable medium of claim 15, whereinwriting the stripe comprises writing the stripe across a word line ineach of the two or more layers.
 17. The non-transitory machine-readablemedium of claim 16, wherein writing the stripe across a word linecomprises: writing a data value of the stripe into a most significantbit page of the word line and writing a subsequent data value of thestripe into a least significant bit page of the word line.
 18. Thenon-transitory machine-readable medium of claim 15, wherein the methodfurther comprises: responsive to writing a final data value of thestripe, performing a parity calculation operation with the final datavalue and the current value of the corresponding parity page and storinga result as the current value in the corresponding parity page; andwriting the current value in the corresponding parity page to the parityvalue of the stripe.
 19. The non-transitory machine-readable medium ofclaim 15, further comprising: writing one or more subsequent stripes ofdata across the two or more layers of each of the one or more dies, eachof the one or more subsequent stripes having one or more data values anda parity value.
 20. The non-transitory machine-readable medium of claim15, wherein a residual bit error rate (RBER) of each of the two or morelayers is averaged across the one or more data values in the stripe, andwherein the average RBER is lower than a worst case RBER.